Efficient Methods for Natural Language Processing: A Survey

Abstract Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows. Such resources include data, time, storage, or energy, all of which are naturally limited unevenly distributed. This motivates research into efficient methods require fewer achieve similar results. survey synthesizes relates current findings NLP. We aim provide both guidance for conducting NLP under resources, point towards promising directions developing more methods.